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Abstract 

Bilinear inverse problems (BIPs), the resolution of two vectors given their image under a bilinear mapping, 
arise in many applications. Without further constraints, BIPs are usually ill-posed. In practice, properties 
of natural signals are exploited to solve BIPs. For example, subspace constraints or sparsity constraints are 
imposed to reduce the search space. These approaches have shown some success in practice. However, there 
are few results on uniqueness in BIPs. For most BIPs, the fundamental question of under what condition 
the problem admits a unique solution, is yet to be answered. For example, blind gain and phase calibration 
(BGPC) is a structured bilinear inverse problem, which arises in many applications, including inverse rendering 
in computational relighting (albedo estimation with unknown lighting), blind phase and gain calibration in sensor 
array processing, and multichannel blind deconvolution (MBD). It is interesting to study the uniqueness of such 
problems. 

In this paper, we define identifiability of a BIP up to a group of transformations. We derive necessary 
and sufficient conditions for such identifiability, i.e., the conditions under which the solutions can be uniquely 
determined up to the transformation group. Applying these results to BGPC, we derive sufficient conditions for 
unique recovery under several scenarios, including subspace, joint sparsity, and sparsity models. For BGPC with 
joint sparsity or sparsity constraints, we develop a procedure to compute the relevant transformation groups. 

We also give necessary conditions in the form of tight lower bounds on sample complexities, and demonstrate the 
tightness of these bounds by numerical experiments. The results for BGPC not only demonstrate the application 
of the proposed general framework for identifiability analysis, but are also of interest in their own right. 

Index terms — uniqueness, transformation group, equivalence class, ambiguity, blind gain and phase cali¬ 
bration, sensor array processing, inverse rendering, SAR autofocus, multichannel blind deconvolution 

1 Introduction 

Whereas linear inverse problems are well-understood and the literature on them is vast, much less is known 

about bilinear inverse problems (BIPs). BIPs, i.e., recovering two variables x and y given a bilinear measurement 
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z = J-{x,y), have attracted considerable attention recently. However, in spite of recent progress, the question 
of identifiability - or uniqueness of the solutions in BIPs under a variety of realistic conditions, has been largely 
open. BIPs arise in many important applications, such as blind deconvolution [^|^, phase retrieval [^|^, 
dictionary learning j^, etc. These problems usually involve recovering the inputs of an under-determined bilinear 
system. They also suffer from scaling ambiguity among other possible ambiguities (e.g., shift ambiguity of blind 
deconvolution, multiplication by a permutation matrix in dictionary learning, multiplication by an arbitrary 
invertible matrix in matrix factorization problems, etc). Therefore, these problems are ill-posed and do not yield 
unique solutions. By introducing further constraints that exploit the properties of natural signals, one can reduce 
the search space, which may help identifiability. For example, cone constraints, such as convex cone constraints 
(e.g., positivity), subspace constraints, and union of subspaces constraints (e.g., sparsity or joint sparsity), are 
very common in BIPs. However, even with a reduced feasible set, a BIP often still exhibits some ambiguities, 
such as scaling [^. 

In this paper, to address the issues of ambiguity, we expand the notion of identifiability of BIPs. We resolve 
the ambiguity issues by allowing uniqueness up to a group of transformations, which define equivalence classes of 
solutions. We then derive necessary and sufficient conditions for identifiability in BIPs up to the transformation 
group. We demonstrate the utility of this proposed framework and the new general results by applying them 
to a prototypical BIP - the blind gain and phase calibration (BGPC) problem, with subspace, or with sparsity 
constraints. The results include the first algebraic sample complexity conditions for this problem. Owing to the 
ubiquitousness of BGPC, the results obtained here for this problem are of broad interest in their own right. 

Blind gain and phase calibration (BGPC) is a bilinear inverse problem that arises in many applications. It is 
the joint recovery of an unknown gain and phase vector A and signal vectors (j>i,(j) 2 , -'' given the entrywise 
product Y = diag(A)$, where <I> = [(j)i, (j) 2 , ■ • • , In inverse rendering [^, when the surface profile (3D model) 
of the object is known, the joint recovery of the albedcQ and the lighting conditions is a BGPC problem. In 
sensor array processing [^, if the directions of arrival of source signals are properly discretized using a grid, and 
the sensors have unknown gain and phase, the joint recovery of the source signals and the gain and phase of the 
sensors is a BGPC problem. In multichannel blind deconvolution (MBD) with the circular convolution model, 
the joint recovery of the signal and multiple channels is a BGPC problem. In all these problems, it is common 
to impose subspace, joint sparsity, or sparsity constraints on the signals represented by the columns of $. 

In this paper, after deriving general necessary and sufficient conditions for identifiability in a BIP up to the 
transformation group, we apply these to BGPC and give identifiability results under several scenarios. We first 
consider a subspace constraint and provide an alternative proof for the result in inverse rendering [^. Then 
we consider a joint sparsity constraint. We develop a procedure to determine the relevant equivalence classes 
and transformation groups for different bases. Then we give sufficient conditions for the identifiability of jointly 
sparse signals (ID or 2D), or piecewise constant signals. For BGPC with subspace or joint sparsity constraints, 
we also give necessary conditions in the form of tight lower bounds on sample complexities. We show that the 
sufficient conditions and the necessary conditions coincide in some cases. We design algorithms to check the 
^Albedo, also known as reflection coefficient, is the ratio of reflected radiation from a surface to incident radiation upon it. 
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identifiability of given signals and demonstrate the tightness of our sample complexity bounds. We analyze 
the gaps and present conjectures about how to bridge them. Then we derive a universal sufficient condition for 
BGPC with a sparsity constraint. This condtion is the most stringent, but applies to all bases and all equivalence 
classes of solutions. Once the condition is met, the solution of the BGPC problem can be recovered uniquely up 
to an unknown generalized permutation, regardless of the basis. 

The rest of the paper is organized as follows. We introduce the problem setup and compare our results 
with related work in the rest of this section. In Section we formally define “transformation group” and 
“identifiability” for bilinear inverse problems and derive sufhcent and necessary conditions for identifiability up 
to a transformation group. We derive sufficient conditions for the identifiability of BGPC with a subspace 
constraint, a joint sparsity constraint, and a sparsity constraint, in Sections ii and 1^ respectively. We also 
give necessary conditions and analyze the tightness of our sample complexity bounds in Sections and We 
conclude this paper in Section with some discussion and open problems. 


1.1 Notations 

Before proceeding to the problem statement, we state the notations that will be used throughout the paper. We 
use upper-case letters A, X and Y to denote matrices, and lower-case letters to denote vectors. The diagonal 
matrix whose diagonal entries are the entries of vector A is denoted by diag(A). We use I to denote the identity 
matrix and F to denote the normalized discrete Fourier transform (DFT) matrix. Unless otherwise stated, all 
vectors are column vectors. The dimensions of all vectors and matrices are made clear in the context. A vector 
is said to be non-vanishing, if all its entries are nonzero. 

We use Vlx,^y to denote subsets of vector spaces X,y. The Gartesian product of two sets is denoted by 
X fiy. An element of x fly is denoted by {x,y), where x G fix and y G fly. We use ^x and ^y 


to denote transformation groups (to be defined in Section 2.11. The Gartesian product of two transformation 


groups ^x, S/'y (also known as direct product in group theory terminology) is denoted by SAx x S/y. Elements 
of the transformation groups are denoted hy Tx G SAx^ Ty G SAy and iJ'x^Ty) G SAx x SAy. 

We use k to denote indices, and J, K to denote index sets. If a matrix or a vector has dimension n, 
then an index set J is a subset of {1,2, • • • ,n}. We use |J| to denote the cardinality of J, and J'^ to denote its 
complement. We use superscript letters to denote subvectors or submatrices. Thus, represents the subvector 
of X consisting of the entries indexed by J. The scalar x^^'^ represents the jth entry of x. The submatrix 
has size | J| x |Ar| and consists of the entries indexed hy J x K. The vector A^'-AI represents the fcth column of 
the matrix A. The colon notation is inherited from MATLAB. 

We use ./ and © to denote entry wise division and entry wise product, respectively. Gircular convolution is 
denoted by @. The direct sum of two subspaces is denoted by ©. The kronecker product of two matrices is 
denoted by ©. The row space and column space of a matrix are denoted by Tl{-) and C(-), respectively. 
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1.2 Problem Statement 


We formally state the general bilinear inverse problem (BIP) and a special BIP termed blind gain and phase 
calibration (BGPC) in this section. First, a bilinear mapping is defined as follows. 

Definition 1.1. Let X, y and Z he three linear vector spaces. A bilinear mapping is a function X : X x y ^ Z 
such that for any y € y the mapping x i—^ iF{x,y) is a linear mapping from X to Z and for any x G X the 
mapping y i— J-{x, y) is a linear mapping from y to Z . 

Given the measurement z = iF{xQ,yo), the following feasibility problem is called the unconstrained bilinear 
inverse problem: 


(Unconstrained BIP) find {x,y) S df x 

s.t. X{x, y) = z. 

Bilinear inverse problems are usually underdetermined, and hence do not yield unique solutions. A variety 
of constraints x € C X,y G Lty C y can be imposed to reduce the search space and make the problem 
better-posed. The constrained bilinear inverse problem is: 

(Gonstrained BIP) find {x,y), 

s.t. X{x,y) = z, (1) 

X G Llx, y G Uy. 

Blind gain and phase calibration (BGPC) is the following constrained BIP given the measurement Y = 

diag(Ao)$o: 


find (A, <i>), 
s.t. diag(A)$ = Y, 

A e Ua, $ e 

where A G fix C C” is the unknown gain and phase vector, $ G 12$ C is the signal matrix. In this 

paper, we impose no constraints on A, i.e., fix = C”. As for the matrix $, we impose subspace, joint sparsity, 

or sparsity constraints. In all three scenarios, $ can be represented in the factorized form $ = AX, where the 

columns of A € form a basis or a frame (an overcomplete dictionary), and X G fix C is the matrix 

of coordinates. The constraint set becomes 12 $ = {$ = AX : X G fix}- Under some mild condition^ on A, 

the uniqueness of $ is equivalent to the uniqueness of X. For simplicity, we treat the following problem as the 

^Under a subspace constraint, A is required to have full column rank. Under a joint sparsity or sparsity constraint, A is required to 
satisfy the spark condition [^. 
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BGPC problem from now on. 


(BGPC) find (A,X), 

s.t. diag(A)AX = F, 

A e C”, X e nx- 

Next, we elaborate on the three scenarios considered in this paper: 

(I) Subspace constraints. The signals represented by the columns of $ reside in a low-dimensional subspace 
spanned by the columns of A. The matrix A is tall (n > m) and has full column rank. The constraint set is 

= £mxN^ 

In inverse rendering j^, the columns of F = diag(A)$ represent images under different lighting conditions, 
where A represents the unknown albedo^ and the columns of $ represent the intensity maps of incident light. 
The columns of A are the first several spherical harmonics extracted from the 3D model of the object. They 
form a basis of the low-dimensional subspace in which the intensity maps reside. 

Multichannel blind deconvolution (MBD) with the circular convolution model also falls into this category. 
The measurement = diag(A)<i>^'’'^^ can be also written as: 

= ^(F*A) ® 

Vn 

The vector A represents the DPT of the signal, and columns of $ represent the DPT of the channels. The 
columns of F* A form a basis for the low-dimensional subspace in which the channels reside. Por example, when 
the multiple channels are PIR filters that share the same support J, they reside in a low-dimensional subspace 
whose basis is F*A = , By symmetry, the roles of signals and channels can be switched. In channel encoding, 

when multiple signals are encoded by the same tall matrix E, they reside in a low-dimensional subspace whose 
basis is F*A = E. In this case, the vector A represents the DPT of the channel. 

(II) Joint sparsity constraints. The columns of $ are jointly sparse over a dictionary A, where >1 is a square 
matrix (n = m) or a fat matrix (n < m). The constraint set ^x is 

Vlx = {X G : X has at most s nonzero rows}. 

In other words, the columns of X are jointly s-sparse. 

In sensor array processing with uncalibrated sensors, the vector A represents unknown gain and phase for the 
sensors, and the columns of $ represent array snapshots captured at different time instants. If the direction of 
arrival (DOA) is discretized using a grid, then each column of A represents the array response of one direction 
on the grid. With only s unknown sources, each column of $ is the superposition of the same s columns of A. 
Hence the columns of the source matrix X are jointly s-sparse. 

In synthetic aperture radar (SAR) autofocus |I0| , which is a special multichannel blind deconvolution problem, 
^In inverse rendering, albedos are real and positive. We ignore this extra information here for simplicity. 
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X represents the SAR image and A = F is the ID DFT matrix. The entries in A represents the phase error in 
the Fourier imaging data, which varies only along the cross-range dimensiorj^ If we extend the coverage of the 
image by oversampling the Fourier domain in the cross-range dimension, the rows of the image X corresponding 
to the region that is not illuminated by the antenna beam are zeros. Thus, the SAR image X can be modeled 
as a matrix with jointly sparse columns. 

(Ill) Sparsity constraints. The matrix $ is sparse over a dictionary A, where A is a square matrix (n = m) 
or a fat matrix (u < m). The constraint set is 

Vtx = {X G : X has at most s nonzero entries}. 

A matrix X with sparse columns can be considered as a special case of this scenario. 

Consider the following multichannel blind deconvolution problem. An acoustic signal is transmitted under 
reverberant conditions and recorded by a microphone array. The DFT of the signal is A, A = F is the DFT 
matrix, each column of <i> = AX is the DFT of the channel of a corresponding microphone, and the corresponding 
column of A is a sparse multipath channel that contains nonzero values at a few locations. 

In the rest of this paper, we will address the identifiability of the general BIP and the above BGPC problem. 
For BGPC, the constraint sets fix and fix are closed under scalar multiplication. For any nonzero scalar cr, 
the pairs (Ao,Ao) and (trAo, yAg) map to the same Y and hence are non-distinguishable. We say that this 
problem suffers from scaling ambiguity. The set {(ctAq, yAg) : A S AIa, A £ fix, o' ^ 0} is an equivalence class 
of solutions generated by a group of scaling transformations. More complex ambiguities and equivalence classes 
will be analyzed later. Our identifiability results answer the question under what conditions the solution (Ag, Ag) 
is unique up to scaling, or up to other transformation groups. 


1.3 Related Work 


Recently, solving bilinear or quadratic inverse problems with the methodology of “lifting” has attracted much 
attention. Examples include recent works on blind deconvolution m and phase retrieval [l2}|l4] . In the lifting 
framework, for any bilinear mapping F : C™ x C" —t Z, there exists a linear operator Q : —>■ Z such that 

Q{xy"^) = F{x,y). Given the measurement z = Oixop^) = F(a;g,yg), one can recast the BIP as the recovery of 
the rank-1 matrix xoyj £ fl^d = {xy"^ : x £ fix, y £ 


(Lifted BIP) find M, 

s.t. G{M) = z, 
M £ flM- 


"^In SAR autofocus, the entries of the phase error A have unit moduli. We ignore this extra information here for simplicity. 
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Choudhary and Mitra adopted this framework, and showed that the lifted BIP has a unique solution Mq 
XQ yQ if the null space of Q does not contain the difference of Mq and any other matrix in i.e., 

Af{g) f]{Mo -M:Me = {0}. 


The identifiability analysis hinges on finding the set of rank-2 matrices in the null space of Q. They addressed 
the question of identifiability in an abstract BIP under the assumptions that the set of rank-2 matrices in JV{G) 
has low complexity (e.g., finite cardinality or small covering number). Using this framework, they showed that 
blind deconvolution with a canonical sparsity prior is not identifiable |15| . 

In contrast, we create a more general framework for the identifiability of BIPs. We consider bilinear mappings 
defined on general vector spaces (not just Euclidean spaces). Besides scaling ambiguity, our framework allows 
other ambiguities. We extend the notion of identifiability to identifiability up to transformation groups. Our 


framework is amenable to BIPs with matrix multiplications, such as dictionary learning 16 -20 and the BGPC 


problem. For the BGPC problem, we are able to derive identifiability results under subspace, joint sparsity, or 
sparsity constraints within our framework. Furthermore, we provide an explicit enumeration-based scheme to 
determine, under subspace or joint sparsity constraints, the identifiability of a solution for given measurements. 
No such results are available within the lifting framework: under the same constraints, it is not obvious how to 
find the set of rank-2 matrices in M{g) within the lifting framework. Moreover, since the set of rank-2 matrices 
in M{g) may be infinite, it is also unclear how to check the identifiability condition for any given scenario. 

Other related work has to do with instances of the BGPC problem. The structure of the BGPC problem 
arises in many signal processing applications. In each of these, the problem formulation and treatment were 
tailored to the application. Instead, we address the identifiability of all these problems within the one common 
framework. Nguyen et al. showed a sufficient condition for unique inverse rendering, which falls into the 
category of BGPC problems with subspace constraints. By examining the problem in our framework, we are 
able to replicate Nguyen’s result and provide an alternative proof. In addition, we give a new necessary condition 
that features a tight lower bound. Morrison et al. proposed an algorithm for SAR autofocus and showed 
a necessary condition for their algorithm. If the support is unknown, the SAR autofocus problem falls into 
the category of BGPC problems with joint sparsity constraints. Using our notion of identifiability up to a 
transformation group, we provide a sufficient condition for unique recovery up to an unknown scaling and a 
circular shift. Most works on the identifiability of MBD considered the linear convolution model [2,21 . These 
traditional works used finite impulse response (FIR) models, and never incorporated joint sparsity, or sparsity. In 
contrast, we consider the circular convolution model, which is more challenging in that the circular convolution 
with a vector can be non-injective, while the linear convolution with a vector is always injective. On the other 
hand, the circular convolution model is more general. By zero padding the signal and the channels (equivalent 
to Fourier domain oversampling), linear convolutions can be rewritten as circular convolutions with a support 
constraint. That falls into the category of BGPG with a subspace constraint. As an important extension of the 
theory of MBD, we study in this paper MBD with subspace, joint-sparsity, and sparsity constraints. 
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2 Identifiability of Bilinear Inverse Problems 


2.1 Transformation Groups and Equivalence Classes 

An important question concerning a bilinear inverse problem is to determine when it admits a unique solution. 
To formulate a good answer, we need to be able to handle the ambiguities of a bilinear inverse problem. For any 
nonzero scalar a such that axQ G VLx and S Vly, by bilinearity, F{ax{), yj/o) = = z. Therefore, the 

constrained BIP does not yield a unique solution if contain such scaled versions of Xq, yo- That is called 

scaling ambiguity. When fix, are closed under scalar multiplication (e.g., subspaces or unions of subspaces), 
the set [(xo,j/o)] = {(cxq, y?/o) : xq G fix, Vo G fly, a ^ 0} is an equivalence class with an exemplar (xo,yo)- 
The transformation 7~ '■ fix x fix f^y such that T~{x,y) = {ux, ^y) is an equivalence transformation. 

The set of all such transformations 

= {T : T{x,y) = {ax, —y), for some nonzero tr G C} (2) 

a 

forms a transformation group. In group theory terminology, the equivalence class [(xq, j/o)] is the orbit of (xq, j/o) 
under the action of ^ |22| . Any valid definition of unique recovery must include uniqueness up to scaling, i.e., the 
equivalence class [(xo,j/o)] can be uniquely identified. There can be other ambiguities for a particular bilinear 
inverse problem (e.g., shift ambiguity of blind deconvolution). We need formal definitions of transformation 
groups and equivalence classes before proceeding towards identifiability. 

Definition 2.1. A set SAx of transformations from fix to itself is said to be a transformation group on fix, if 
the following properties hold: 

1. For any Tx,i,Tx ,2 £ -f^x, the composition of the two transformations Tx ,2 °Tx,i belongs to SAx- 

2. SAx contains identity transformation lx{x) = x for all x G fix- 

3. For any Tx G 3^x, there exists Tff^ G S^x such that Tff^ oTx = Tx ° Tff^ = lx- 

If 37x, SAy are transformation groups on fix, fly respectively, then their direct product .f/'x x f^y is a trans¬ 
formation group on fix x fly. The action of {Tx,Ty) G f^x x fJy on (x, y) G fix x fly is {Tx{x),Ty{y)). If there 
exists T = {Tx,Ty) G i^x x fFy, such that 

F{T{x,y)) = F{Tx{x),Ty{y)) = F{x,y), 

for all (x, y) G fix x fly, then T maps a pair (x, y) to another pair {Tx{x),Ty{y)) so that the two pairs cannot be 
distinguished by their images under iF. If a set of such T’s form a subgroup of .i^x x we have a transformation 
group associated with the bilinear mapping T. 

Definition 2.2. A transformation group 3F on fix x fly is said to be a transformation group associated with 
the bilinear mapping T if: 

1. fF G A^x X 37y is a subgroup of the direct product of two transformation groups fFx o-nd fFy, on fix und 


fly, respectively. 


2. For all (x,y) G Q.x x Vty and for all T G SF, F{x,y) = F{T{x,y)). Or equivalently, F = F oT for all 


TG^. 


To enable an identifiability result up to a transformation group (see Section 2.21, the transformation group 
must capture all inherent ambiguities of the BIP. This motivates the following definition of the ambiguity 
transformation group of the bilinear mapping. 


Definition 2.3. A transformation group SA on x fly is said to be the ambiguity transformation group 
of the bilinear mapping F if !F is the largest transformation group associated with F, i.e., if contains all 
transformation groups associated with F. A transformation F in the ambiguity transformation group F of the 
bilinear mapping F is said to be an equivalence transformation associated with F. 

Next, we define an equivalence class associated with the bilinear inverse problem. 


Definition 2.4. Given the ambiguity transformation group !F of the bilinear mapping F on Fix x Fly, and 
{xo:yo) G Fix X Fly, the set 


[{xo,yo)] 3 r = {{x,y) G Fix x Fy : {x,y) = T{xo,yo) for some T G F'} 

is called the equivalence class of {xo,yo) associated with the bilinear inverse problem m Q. In group theory 
terminology, [(a^oi 2/o)].y is called the orbit of{xo,yo) under the action of F. 

Definition 2.5. Given the ambiguity transformation group .F of the bilinear mapping F on Fx x Fy, and 
Xo G Fx, the set 

[xo]^ = {x gFx : ^yo,y G Fy, s.t. {x,y) G [(a;o,yo)].:^} 
is called the left equivalence class of xq. 

Similarly, given the ambiguity transformation group SA of the bilinear mapping F on Fx x Fy, and yo G Fy, 
the set 

[yo]§~ = {y GFy : 3xo,x G Fx, s.t. {x,y) G [(a;o,yo)]^"} 
is called the right equivalence class o/yg. 

The definition of a transformation group guarantees that the relation between elements in an orbit satisfies 
reflexivity, transitivity and symmetry. Therefore, an orbit is an equivalence class. If F is the ambiguity 
transformation group of the bilinear mapping F, then all the elements in the equivalence class [{xQ,y(f)\^ share 
the same image under F. Therefore, they are equivalent solutions to the bilinear inverse problem in Q. In fact, 
under some mild conditions on the bilinear mapping. Definitions |2.2| and |2.3| have additional implications. 

Proposition 2.6. Assume that the bilinear mapping F has no non-trivial left annihilator of Fy, i.e., if 
F{xo,y) = 0 for all y G Fy, then xq = 0. Then every equivalence transformation F = {Fx,Fy) G -F sat¬ 


isfies the following: 

• If 0 G Fx, then FxiO) = 0. 
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• For xi,X2 G and scalars 01,02, if aiXi +02X2 S fix, then 


Tx{aixi + 02X2) = aiTx{xi) + a 2 Tx{x 2 )- 

If fix is a linear vector space, then Tx is a linear transformation. 

Similarly, assume that the bilinear mapping J- has no non-trivial right annihilator of fix, i.e., if IF{x,yQ) = 0 
for all X G fix, then yo = 0. Then every equivalence transformation T = {'Tx,'Ty) G satisfies the following: 

• If 0 G fly, then 7y{0) = 0. 

• For yi,y 2 G fly and scalars 61 , 62 , if biyi + 622/2 G fly, then 

Ty{biyi + 622/2) = biTyiyi) + b2Ty{y2)- 

If fly is a linear vector space, then Ty is a linear transformation. 

Proof. Due to the symmetry, we only need to prove the results for Tx- 

If 0 S fix, then T{Tx{0),y) = ■^(7”(0,7^^(2/))) = T{0,Ty^{y)) = 0 for all y G fly. By assumption, there is 
no non-trivial left annihilator of fly. Therefore, Tx(0) = 0. 

If OiXi -I- O 2 X 2 G fix, then 


T(Tx(aiXi +a 2 X 2 ),y) 

=T(T(aiXi + 02 X 2 ,Ty^y))) 

=T(aiXi + 02 X 2 ,Ty\y)) 

=aiT(xi,Ty\y)) + a2T(x2,Ty\y)) 

=aiT(Tx(xi),y) + a2T(Tx(x2),y) 

=T(aiTx(xi) + a 2 Tx(x 2 ), y). 

Then TifTxi.o-iXi + 02 X 2 ) — {aiTx{xi) + a 2 Tx(x 2 )),y) = 0 for all y G fly. There is no non-trivial left annihilator 
of fly. Hence 7)r(oiXi - 1 - 02 X 2 ) = aiTxixi) + a 2 Tx{x 2 ), and Tx is a linear transformation if fix is a linear vector 
space. □ 

Bilinear mappings that arise in applications usually have no non-trivial left or right annihilators. Therefore, 
common equivalence transformations, such as scaling and shift, are linear transformations. However, there are 
examples where equivalence transformations are nonlinear (cf. Appendix [A| . 

Before proceeding to identifiability, let us consider the following blind deconvolution problem as a concrete 
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example. The measurement is z = xq ® yo ^ C". 


find {x,y), 
s.t. X ®y = z, 

xGC'^, yG C”. 


Define transformation groups on Af = y = C": 


yy = = {7c" : 7c"(x) = aSi{x), for some nonzero cr S C and some integer £}, 


where the linear transformation is the circular shift by £, defined as follows. If a; = Si{xq), then x^^'^ = Xq 
for all 1 < j,k < n where j — k = £ (modulo n). Then the following subgroup C ^x x ^y is a transformation 
group associated with circular convolution: 


= i T : T{x,y) = ( aSe{x), -S-e{y) 


for some nonzero a G C and some integer £ 


(3) 


Note that y' is a transformation group associated with circular convolution, and a subgroup of ^x x ^y. 
However, it is not separable, i.e., it cannot be written as the direct product of two transformation groups. 
Furthermore, y' is not the ambiguity transformation group, because it does not capture all the ambiguities of 
the above blind deconvolution problem. For example, there exist non-trivial vectors u,v G C" such that u® v is 
the kronecker delta. Thus, {x ® u,y ® v) is an equivalent pair of {x,y). The set of such transformations is not 
contained in y. 


2.2 Identifiability up to a Transformation Group 

The concept of identifiability should be generalized to allow unique recovery up to the ambiguity transformation 
group. If the equivalence class containing the solution can be uniquely identified, the solution is considered 
identifiable. 


Definition 2.7. In the constrained BIP, the solution (xo,yo) in which Xq ^ 0,yo ^ 0 is said to be identifiable 
up to a transformation group S', if every solution {x,y) satisfies that (x,y) = T{xo,yo) for some T G y, or 
equivalently, {x,y) G [(xq, 


In general, the ambiguity transformation group for a certain BIP may not be known a priori. It may require 
some insight to capture all the ambiguities inherent in the problem. However, we can tell whether or not a 
given transformation group is the ambiguity transformation group by checking the identifiability. If there exists 
an identifiability result up to this transformation group, it has to be the largest. If the constraint sets fix and 
fly are closed under scalar multiplication, then one can start by checking the group of scaling transformations 
defined in ([^. For some BIPs, the ambiguities go beyond scaling ambiguity. Hence we have to choose larger 


transformation groups. An example is BGPC with a joint sparsity constraint (Section 4.11. 


11 



We derive a necessary and sufficient condition for identifiability in Theorem |2.8| and a more intuitive sufficient 
condition in Corollary |2.9| Here is how we interpret these results: In order to prove that certain conditions are 
sufficient to guarantee identifiability up to a transformation group, it suffices to first show that Xq can be identified 
up to the transformation group; and then show that once xq is identified and substituted in the problem, j/o can 
be identified. By the symmetry of the problem, we can derive another sufficient condition by switching the roles 
of xo and yg. 

Theorem 2.8. In the constrained BIP, the pair {xo,yo) (xq ^ 0,yo ^ 0) is identifiahle up to HZ' if and only if 
the following two conditions are met: 

1. If J^{x,y) = J^{xo,yo), then x G [xq]^. 

2. IfJ^{xo,y) =J^{xo,yo), then {xo,y) G [{xo,yo)]sr. 


Proof. To prove sufficiency, we suppose Conditions 1 and 2 are met. Let iF{x,y) = iF{xo,yo) for nonzero 
Xo,yo- Then, by Condition 1, x G [ xq ]^- Hence, there exists 71 = G ^ such that x = Tx.i{xo). 

Therefore iF{xo,yo) = ^{x,y) = J-{Tf~^{x,y)) = F{xQ,Ty\{'y)). By Condition 2, there exists 71 G .17 such that 
{xo,Tyl{y)) = 71(xo,yo)- Hence {x,y) = Ti{xQ,Ty\{y)) = 71 o71(xo,yo), and (xo,yo) is identifiable up to . 


Next we prove necessity. Given that (xo,j/o) {xq 0,yo 0) is identifiable up to y7, by Definition 2.7 if 
iF{x,y) = J^{xo,yo), then {x,y) G [(xo,yo)]5^- The necessity of Conditions 1 and 2 follows. □ 


Corollary 2.9. In the constrained BIP, the pair (xo,yo) (xo y^ 0, j/o y^ 0^ is identifiahle up to 17 if the following 
two conditions are met: 

1. If 7{x,y) = 7{xo,yo), then x G [xq]^. 

2. If 7{xo,y) = 7{xo,yo), theny = yo. 

Furthermore, if iP has no non-trivial right annihilator of fix, o,nd for {'Tx,Ty) G 7, Tx^Xq) = Xq only if 
Tx = ^x, then the sufficient conditions above are also necessary. 


Proof. Given that y = yo, we have that {xQ,y) = l(xo, 2 /o) and hence {xo,y) G [{xQ,yo)\sr. Therefore, condition 
2 in Corollary |2.9| is more demanding than that of Theorem |2.8[ Sufficiency follows. 

The necessity of condition 1 also follows from Theorem |2.8| Next we show that with the extra assumptions. 


condition 2 is also necessary. Given that (xo,yo) {xq yf 0,yo y^ 0) is identifiable up to 7, by Theorem 2.8 if 
P{xo,y) = P{xo,yo), then there exists T = {Tx,7y) G such that (xo,y) = T{xo,yo). The first argument 
Tx{xo) = 2 ^ 0 , by the extra assumption, Tx = lx- Now, for all (xi,j/i) G i^x x ^y, T{xi,yi) = T{T{xi,yi)) = 
T{lx{xi),Ty{yi)) = T{xi,Ty{yi)), or equivalently, T(xi,yi — Ty{yi)) = 0. By the extra assumption that T 
has no non-trivial right annihilator oi Fix, Vi — Ty{yi) = 0 for all yi G fly, or equivalently, Ty = 1y. Therefore, 
y = Ty{yo) = yo, and condition 2 is necessary. □ 


The extra assumptions in Corollary |2.9| are usually satisfied, which means that Condition 2 is usually also 
necessary. Indeed, most bilinear mappings that arise in applications have no non-trivial annihilators. The 
assumption that ^‘Txixo) = xq only if Tx = lx” is also true in many scenarios. For example, if Tx is scaling by 
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a nonzero complex number and Tx{xo) = xq for some nonzero xq, then 7x has to be identity. However, there 
are examples for which Corollary |2.9| is not necessary (cf. Appendix [A| . 

Later in this paper, we repeatedly apply Corollary |2.9| to various scenarios of the blind gain and phase 
calibration problem and derive sufficient conditions for identifiability up to transformation groups. 

3 BGPC with a Subspace Constraint 

In this section, we consider the identifiability of the BGPC problem with a subspace constraint. The measurement 
in the following problem is T = diag(Ao)AAo. The known matrix A £ jg tall (n > m). The columns of 

$ = AX reside in a low-dimensional subspace. The constraint sets are JIa = C" and ilx = hence the 

problem in unconstrained with respect to A and X. 

find (A, A), 
s.t. diag(A)AA = Y, 

A G C", A € 


3.1 Sufficient Condition 

As was mentioned earlier, the BGPC problem suffers from scaling ambiguity. The ambiguity transformation 
group is defined as follows: 

^ = {T : T(A, A) = (crA, —A), for some nonzero a G C}. (4) 

cr 

Next, we investigate identifiability up to scaling within the framework of Section By applying Corollary 


lemma (See Appendixfor the proof). 

Definition 3.1. The row space of a matrix A G is said to he decomposable if there exists a non-empty 

proper subset (neither the empty set nor the universal set) J C {1,2, •• • ,n} and its complement J'^ such that 

n{A) = 7^(A(A0) 0 

Lemma 3.2. 1. If A has full row rank, then the row space of A is decomposable. 

2. If A G column rank and its row space is not decomposable, then n > m. 

3. The row space of A is not decomposable if and only if dim{IZ{A)) < dim(72.(A*^'^’'^)) -|-dim(72.(A('^ ’!)) for 
all non-empty proper subsets J C {1, 2, • • • , u}. 

Nguyen et al. referred to the property that “A has full column rank and its row space is not decomposable” 
as “nonseparable full rank”. Here is our restatement of the identifiability result followed by an alternative proof. 

Theorem 3.3. In the BGPC problem with a subspace constraint, the pair (Ag, Aq) G C" x is identifiable 

up to an unknown scaling if the following conditions are met: 


2.9 we provide an alternative proof for the results by Nguyen et al. [H. We need the following definition and 
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1. Vector Ao is non-vanishing, i.e., all the entries of Xq are nonzero. 

2. Matrix Xq has full row rank. 

3. Matrix A has full column rank and its row space is not decomposable. 

Proof. We apply Corollary |2.9| to the BGPC problem, and verify that the two conditions in the corollary are 
satisfied. First, since the vector Aq is non-vanishing and the matrix A has fnll column rank, diag(Ao)^ has full 
column rank. It follows that if diag(Ao)^Xo = diag(Ao)AXi, then Xi = Xq. Hence, given Aq, the recovery of 
Xq is unique. This verifies Condition 2 in Corollary |2.9[ To verify Condition 1, we only need to show that Aq is 
identifiable up to scaling. 

We prove by contradiction. Suppose the opposite, that there exists (Ai,Xi) such that diag(Ao)AJfo = 
diag(Ai)HXi but Ai ^ [Aq]^. Recall that all the entries of Aq are nonzero, A has full column rank and Xq has 
full row rank. Therefore, rank(diag(Ao)AWo) = rank(diag(Ai)AlXi) = to, and Xi too has full row rank. Since 
the row space of A is not decomposable, there are no zero rows in A. Because Xq and Xi have full row rank, 
it follows that there are no zero rows in AXq or AXi. The vector Aq is non-vanishing, hence Ai too is non¬ 
vanishing. Let 7 = Ai./Ao denote the entrywise ratio of Ai over Aq, where 7 ^^^ = 7 ^ OjJ = 1i2, •• • ,n. 

By the assumption that Ai ^ the entrywise ratio is not the repetition of the same number, i.e., there 

exist ji,j 2 such that ^ Let T denote the number of distinct values of Create a partition of the 

index set {1, 2, • • • ,n}, denoted by Ji, J 2 , ■ ■ ■ , Jt, such that 7 *^t) = for all j G Jt, t = 1,2, ■■■ ,T. Note that 
7ii 72 , • • • , 7t are the distinct values of 
Consider the row spaces of A: 

T 

nw = E (5) 

Denote the dimension of by to*. Then there exists a subset C Jt such that \J^ \ = mt and the rows 

of form a basis of TZ{A^‘^*^’-l). By the condition that the row space of A is not decomposable, the sum in 

(|^ is not a direct sum, hence to = rank(H) < Furthermore, by ([^, there exists a subset 

T T 

j'" = {31,32,-■■ ,3m} ^ U Jt = {1,2,-•• ,n} 

such that I J^l = TO and the rows of A^'^’’’-'> form a basis of 71(A). The set (IJ^i ^t) \ is not empty because 
TO < Without loss of generality, we may assume that there exists jo G J{ \ J^. The row 

can be written as a linear combination of the rows of A^-^ ’'i and the representation is unique. We denote the 
representation by: 

^Oo.:) _ -l- • • • -l- aj^A^^'^'''\ (6) 

The rows of are linearly independent, and jo ^ JiC\ J^, hence cannot be written as a linear combi¬ 
nation of the rows of there exists at least one nonzero term in the representation corresponding 

to one of the rows of A^-^ ^ Thus, without loss of generality, there exists ji S P| J|, such that Ojj 7 ^ 0. 

Recall that rank(diag(Ao)AXo) = rank(diag(Ai)7lXi) = to, and Xq and Xi have full row rank to. Therefore, 
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the column spaces satisfy: 


C(diag(Ao)A) = C(diag(Ao)AXo) = C(diag(Ai)y4Xi) = C(diag(Ai)A). 


Hence rank([diag(Ao)A, diag(Ai)H]) = m. Defining matrix 


B := [H,diag( 7 )H] = [diag(Ao)] ^[diag(Ao)A,diag(Ai)H]. 


We have that 


rank(i?) = rank([diag(Ao)A, diag(Ai)H]) = m. 


(7) 


Then we consider the row spaces of B. The dimension of the row space 71{B^'^*’'^) = is also 

rut, and the rows of B^'^t’''> form a basis of the above row space. The rows of form a linearly independent 

set of cardinality m. By 0. the rows of B^'^ '■'> form a basis of TZ{B). The row can be can be written as 

a linear combination of the rows of B^'’'’''\ We denote the representation by: 


+ • • • + ( 8 ) 

Recall that jo £ Jf, ji £ J|, hence = 71 , and 7 (^ 1 ^ = 72 . Using the definition of B, we rewrite Q as: 





(9) 

AUoA 

= /3jj72A^'’1’'^ -I- -|- • • 


(10) 


Since the representation in ([^ is unqiue, the representations in ([^ and (10) must satisfy: 


~ Ph ~ 

7i 


It follows that 7 i = 72 , which contradicts the assumption that 71 and 72 are distinct. Hence the assumption 
that Ai (f: [Aq]^ is false, and Aq is identifiable up to an unknown scaling. □ 


For generic signals, we can show that Theorem 3.3 reduces to a simple condition (Corollary 3.4) on the 
dimensions n, m and TV. We say that a property holds for almost all signals if the property holds for all signals 
but a set of measure zero. 


Corollary 3.4. In the BGPC problem with a subspaee eonstraint, if n > m and N > m, then (Ao,Rlo) ** 
identifiable up to an unknown scaling for almost all Aq £ C", almost all Xq £ and almost all A £ 


Proof. Almost all Aq £ C" are non-vanishing. If TV > m, almost all Xg £ have full row rank. If 

n > m, almost all A £ have full column rank. Next we show that the row spaces of almost all A are not 

decomposable. For almost all A, the submatrices A^'^’’^ and A^'^ have full rank for every non-empty proper 
subset J C {1, 2, • • • , n}. Therefore, one of the following cases has to be true. 
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1. If I J| < m and \ J‘^\ < m, then for almost all A, dim(7^(^)) = m, dim(7^(A(‘^’-))) = | J|, dim(7^(A*^‘^''’'))) = 

Hence for almost all A, 

dim(7^(H)) = m < n = \ J\ + | J°| = dim(7?.(H^'^’’^)) + dim(7^(H^'^ ’■^)). 

2. If |J| > m, then for almost all H, dim(7^(H*^'^’-))) = m. Hence for almost all A, 

dim(7^(A)) = m < m + 1 < dim(7^(A('^-))) + dim(7^(A('^°-))). 

3. If I J'^l > TO, then for almost all A, dim(7?,(A^‘^''’'))) = TO. Hence for almost all A, 

dim(7^(A)) = to < 1 + to < dim(7^(A('^-))) + dim(7^(A('^°-))). 

Therefore, dim(7?.(A)) < dim(7^(A('^’-^)) + dim(7^(A(‘^‘'’'))) for every non-empty proper subset J C {1,2, • • • ,n}, 
establishing that the row spaces of almost all A are not decomposable. By Theorem |3.3| given that N > m and 
n > m, the pair (Aq, AIq) is identifiable up to an unknown scaling for almost all Aq, Xq and A. □ 

Corollary |3.4| shows that, in the BGPC problem with a subspace constraint, for almost all vectors Aq, almost 
all tall matrices A and almost all fat matrices Xq, the solution (Aq, Xq) is identifiable up to an unknown scaling. 

3.2 Necessary Condition 

Given that Aq is non-vanishing, Nguyen et al. showed that “the row space of A is not decomposable” is 
necessary. Lacking however, is a necessary condition for the sample complexity. 

As we demonstrate in the next subsection by construction of counter-examples, the sample complexity N > m, 
as required by Theorem |3.3| implicitly and Corollary |3 . 4| explicitly, is not necessary. Instead, a necessary condition 
is suggested by heuristically counting the number of degrees of freedom and the number of measurements in 
Y = diag(A)AA. The numbers of free variables in A and X are n and mN, respectively. The unknown scaling of 
A and X is counted twice, hence 1 is subtracted yielding n + mN — 1 for the total number of degrees of freedom. 
The total number of measurements is nN. Heuristically, to achieve uniqueness, nN must be greater than or 
equal to n + mN — 1, which implies N > . This turns out to be a valid necessary condition, as we now state 

and prove rigorously. 

Proposition 3.5. In the BGPC problem with a subspace constraint, if A has full column rank, and (Ao,Aio) 
(with a non-vanishing Xq) is identifiable up to scaling, then N > ■ 

Proof. We show that if then the recovery cannot be unique. Let A± G £ri^{n-m) ^ matrix 

whose columns form a basis for the ortho-complement of the column space of A. Hence A^ is an annihilator of 
the column space of A. Consider the linear operator Q : C" —>■ defined by 

Q{x) := A^ diag(x)F = A^ diag(a;) diag(Ao)AXo. 
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We claim that every non-vanishing null vector of Q produces a solution to the BGPC problem. Indeed, if 
X e M{G), then 

A*j_ diag(a;) diag(Ao)AXo = 0, 

hence the columns in diag(a:) diag(Ao)AXo must reside in the column space of A. Let 

diag(a;) diag(Ao)AXo = AXi. 

If X is non-vanishing, then (Ai, Xi) is a solution, where Ai is the entrywise inverse of x. 

Let xq denote the entrywise inverse of Ag, then xq € Af{G). There are N(n — m) equations in G(x) = 0. 
If < nZm ’ N{n — m) < n — 2, the dimension of the null space N{Q) is at least 2. Hence, there exists 
another vector x\ S Af{G) such that xo,xi are linearly independent. Let a be a complex number such that 
0 < |a| < —. Then xq -I- axi S Af{Q) is non-vanishing, because the entries of xq -\- axi satisfy that 

-f ax[^^\ > -|a|||xi||oo > 0, for every j e {1,2, - • • ,n}. 

Il^olloo 

This null vector is not a scaled version of xq- Hence there exists a solution that does not belong to the equivalence 
class [(Ao,Xo)]5 ^- Therefore, N > is necessary. □ 

The two sample complexities N > m and N > coincide when TO = lorTO = n—1. The gap between 
these two sample complexities when 2<m<n — 2 is analyzed next. 

3.3 Gap Between the Sufficient and the Necessary Conditions 

The sample complexity in the sufficient condition is N > m, which can be represented by the region to the right 
of a line segment. The sample complexity in the necessary condition is N > which can be represented 

by the region to the right of part of a hyperbola. The gap between the two sample complexities is the region 
between the line segment and the hyperbola (cf. Figure [^. 



N 


Figure 1: The sample complexities for BGPC with a subspace constraint, and the ratio of identifiable 
pairs generated randomly. 

To explore this gap, we wish to determine whether (Ao,Wo), in BGPC with a subspace constraint, is iden¬ 
tifiable up to scaling. We now show that this can be done by Algorithm 1. Given A that has full column rank 


17 











and Y = diag(Ao)AXo that has no zero rows, Algorithm 1 returns a boolean value indicating whether (Aq, Aq) 
is identifiable up to scaling. 


Algorithm 1 Identifiability of the BGPC problem with a subspace constraint 
input: A,y output: identifiability of (Aq, Aq) 

[Q,R]= qr(A) 


A, 


G 


Q{:,m+l:n) 


[diag(y*^'’^^)]*A_L [diag(y*^'’^^)]*Aj 

if rank(G) < n — 2 then 
return False 
else 

return True 
end if 


•• [diag(y(-^))]Mx 


// QR decomposition of A 


Proposition 3.6. Given A that has full column rank and Y = diag(Ao)AAo that has no zero rows, the pair 
(Ao,Alo) is identifiable up to scaling if Algorithm 1 returns True, and not identifiable up to scaling if Algorithm 
1 returns False. 

Proof. The columns of A± form a basis for the ortho-complement of the column space of A, hence A*^ is an 
annihilator of the column space of A. The matrix G G satisfies that Gx = vec(A^ diag(x)y). Given 

y that has no zero rows, any solution to the BGPC problem (A, A) satisfies that A is non-vanishing, and that 
the entrywise inverse of A is a null vector of G. On the other hand, as argued in the proof of Proposition |3.5| 
any non-vanishing null vector of G produces a solution (A, A). 

If Algorithm 1 returns True, then rank(G) > n — 1. Given a solution (Ao,Ao), G has at least one null 
vector mo, which is the entrywise inverse of Aq. Hence rank(G) = n — 1. All the null vectors of G reside in the 
one-dimensional subspace spanned by xq. Therefore A in any solution is a scaled version of Aq, or A S [Aq];^- 
Given non-vanishing Aq and A with full column rank, diag(Ao)A has full column rank and the recovery of Aq 
has to be unique. By Corollary |2.9[ (Aq, Aq) is identifiable up to scaling. 

If Algorithm 1 returns False, then rank(G) < n — 2. By the proof of Proposition |3.5| (Ao,Ao) is not 
identifiable. □ 


We now use Algorithm 1 to construct counter-examples demonstrating that the sufficient condition in The¬ 
orem 


3.3 is not necessary. Let n = 10, 1 < m < 9, and 1 < A < 9. The entries of Aq G M" and Ag G 


2XN 


are 


generated as iid Gaussian random variables A(0,1). The matrix A G jg ^ columns from an n x n 

random orthogonal matrix. Then A± comprises of the last (n — m) columns from the same random orthogonal 
matrix. We use Algorithm 1 to determine whether or not (Ag, Ag) is identifiable up to scaling. For every value 
of m and A, the numerical experiment is repeated 100 times independently. The ratio of identifiable pairs as a 
function of (m. A) is shown in Figure]^ As is expected, the solution (Ag, Ag) is identifiable when A > m, and is 
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not identifiable when N < ■ Meanwhile, when < N < m, the ratio of identifiable pairs is 1. Therefore, 

N > m is not necessary. 

On the other hand, the necessary condition in Proposition |3.5| is not sufficient. For example, if n = 8 , m = 4 
and < N = 2 < m, let A he the structured matrix 


A = 


Ai 


diag(7)Ai , 


where Ai G 7 G C®. There exists an Ai and a 7 such that the matrix A has full column rank and the 

row space of A is not decomposable. For example, let Ai = and 7 = 2\/2F^'''^\ then A = 2\/2F^''^-‘^\ 

However, (Ao,Xo) is not identifiable and XqAXq = XiAXi, if 


1 

1 

0 


1 

0 

1 - 

0 

0 

1 

II 

0 

0 

0 
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1 

0 

1 - 

0 

1 

0 


1 

0 

1 


Aq — 7 © Ai. 


However, according to the ratio of identifiable pairs shown in Figure the unidentifiable case does not occur 
even once in 100 random trials. We have the following conjecture: 

Conjecture 3.7. In the BGPC problem with a subspace constraint, if n > m and N > then (Ao,^o) ** 

identifiable up to an unknown scaling for almost all Aq G C", almost all Xq G and almost all A G 

If the above conjecture is true, the necessary condition N > ffZ^ tight except for a set of measure zero. 


4 BGPC with a Joint Sparsity Constraint 

Here we consider the identifiability in the BGPC problem with a joint sparsity constraint: 

(PI) find (A,X), 

s.t. diag(A)HX = Y, 

X G C", X G Ztx = {X G : the columns of X are jointly s-sparse}. 

The measurement in the above problem is y = diag(Ao)v4Xo. We only consider the case where A G is an 

invertible square matrix. The vector Aq G C" is non-vanishing. The columns of Xq G are jointly s-sparse 

(Xg has at most s nonzero rows). Unless otherwise stated, we assume that the sparsity level s is known a priori. 
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However, if s is unknown, one can solve the following optimization problem instead: 


(P2) min. row-sparsity (X), 

s.t. diag(A)HX = F, 

A e C”, a: G c”^^. 

In this section, we define ambiguities and transformation groups that depend on the matrix A. For two 
special cases of A, we give sufficient conditions for identifiability up to the ambiguity transformation groups. 

4.1 Ambiguities and Transformation Groups 

Geometrically, a joint sparsity constraint corresponds to a union of subspaces; hence, it is less restrictive than 
the previously discussed subspace constraint. This results in greater ambiguity in identifying a solution to BGPC 
with a joint sparsity constraint, than just the scaling ambiguity. In this case, to obtain identifiablity results, we 
must choose the largest transformation group associated with the BIP, which captures all ambiguities inherent 
in the problem. In this section, we develop a procedure to do so. 

A generalized permutation matrix is an invertible square matrix with exactly one nonzero entry in each row 
and each column. It preserves the joint sparsity structure. That is, if the columns of Xq are jointly s-sparse 
and P is a generalized permutation matrix, then the columns of Xi = PXq are also jointly s-sparse. Suppose 
there exists a vector 7 G C" such that P = A~^ diag( 7 )A is a generalized permutation matrix; then clearly 
7 has to be non-vanishing. Now, given a solution (Ao,Ao) to the BGPG problem, there exist Ai = A 0./7 and 
Xi = PXq G such that 

diag(Ai)AAi = diag(Ao)[diag( 7 )]“^AA“^ diag( 7 )AAo = diag(Ao)AAo. 

This ambiguity is inevitable. To address this ambiguity, we define the set 

r(A) = {7 G C" : A~^ diag( 7 )A is a generalized permutation matrix}, (11) 

and the ambiguity transformation group 

= {T : T{X,X) = (A./ 7 , diag( 7 )AA) for some 7 G r(A)}. ( 12 ) 

Then (Ai, Ai) is in the equivalence class [(Aq, Ao)]^". 

Note that the set r(A) depends on A. In particular, when A is the normalized DFT matrix A = F G C”^", the 
matrix F* diag( 7 )P is a circulant matrix whose first column is -^F*j. The matrix F* diag( 7 )P is a generalized 
permutation matrix if and only if there is exactly one nonzero entry in -XF*'y, which means that the circulant 
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matrix F* diag{'y)F is a scaled circular shift. Therefore, 


r(F) = {7 = uy/nF^'''^'^ : cr G C is nonzero, fc G {1, 2, • • • , n}}. 


(13) 


= {T : T{X,X) = {X./'y, F* diag{'j)FX) for some 7 G r(F)}. 


(14) 


An equivalence transformation T G ^ defined in (141 is a complex exponential modulation of A scaled by 7 and 
a circular shift of X scaled by a. In MBD, if we shift the signal by 1 — fc and scale it by i, and shift the channels 
by fc — 1 and scale them by a, the outputs of the channels remain unchanged. 

The ambiguity transformation groups for other choices of A can be figured out in a similar fashion. For more 
examples, please refer to Section [4!^ and to Appendix [C| 


4.2 Identifiability of Jointly Sparse Signals 

In this section, we assume that A = F is the DFT matrix and the columns of X are jointly s-sparse. In 
multichannel blind deconvolution, the non-vanishing vector Aq is the DFT of the signal and the jointly sparse 
columns of Aq are the multiple channels. We derive a sufficient condition and a necessary condition for (Aq, Aq) 


to be identifiable up to the transformation group defined in (141. 


4.2.1 Sufficient Condition 


We can prove a sufficient condition for identifiability up to the transformation group in (14) within the framework 


of Section by again invoking Corollary |2.9| We need the following definition to state this sufficient condition. 

Definition 4.1. The index set J = {ji, J 2 , • ■ ■ , Js} C {1, 2, • • • ,n} is said to be periodie with period i (I being 
an integer such that 0 < £ < n), if J = {ji +£, j2 +£■,'•' ,js + (modulo n). The smallest integer I with this 
property is called the fundamental period. 

The universal set {1,2, •• • ,n} is always periodic with period t {£ being any integer from 1 to n — 1). The 
fundamental period is 1. For n = 10 and s = 4, the set J = {1,2,6, 7} is periodic with fundamental period 5. 
Periodicity has the following property. 

Remark 4.2. If the set J = {ji,j 2 ,''’ ,js} Is periodic with period I, then the complement J°, the flipped 
version Jf = {—ji, —j2, ■ ■ ■ , —js} (modulo n) and the shifted version {ji + fc, j2 + fc, ■ • • + fc} (modulo n) are 

all periodic with period i. 

Here is the sufficient condition for the identifiability of the BGPC problem with DFT matrix and a joint 
sparsity constraint. 

Theorem 4.3. In the BGPC problem with DFT matrix and a joint sparsity constraint at sparsity level s, the pair 


(Aq, Aq) G C" X is identifiable up to the transformation group !X defined in (14) if the following conditions 
are met: 

1. Vector Aq is non-vanishing. 
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2. Matrix Xq has exactly s nonzero rows and rank s. 

3. The joint support of the columns of Xq is not periodic. 


Proof. First, given non-vanishing Aq and the DFT matrix F, the matrix diag(Ao)F’ has full rank. If diag(Ao)FXo = 


diag(Ao)F'Afi, then Xi — Xq. Hence, given Aq, the recovery of Xq is unique. By Corollary 2.9 to complete the 
proof, we only need to show that Ag is identifiable up to the transformation group. 

By assumption, the matrix Xq has rank s and the joint support of the columns of Xq, denoted by J = 
{ji,j2,-” )js}, is not periodic. Given that diag(Ao)FXo = diag(Ai)FXi, we show that Ai G [Ag]^- Now, the 
matrix Xq has s linearly independent columns, diag(Ag)F’ has full rank, hence the corresponding columns of Xi 
are also linearly independent. Without loss of generality, we may assume that Xq and Xi only have s columns, 
which are linearly independent, by removing redundant columns at the same locations in both matrices. Then 
Xq,Xi G have full column rank s and exactly s nonzero rows. Because F has no zero entries, it follows 

that there are no zero rows in FXq or FXi. The vector Ag is non-vanishing, hence Ai is also non-vanishing. We 
know that 

P = F*[diag(Ai)]“Miag(Ag)F (15) 


is a circulant matrix and that Xi = PXq. Let Xq G C®^" denote the pseudo-inverse (also the left inverse) of 
Xq, and Xo± G denote a matrix whose columns form a basis for the ortho-complement of the column 

space of Xq. Since Xq has full column rank s and exactly s nonzero rows indexed by J, we may choose Xq such 
that its nonzero columns are indexed by J, and choose the columns of Xgj^ to be the standard basis vectors 
^ g jcj matrix P as in Xi = PXq satisfies 


P = XiXd + QXo\, 


(16) 


where Q G is a free matrix. Note that the nonzero columns of QXqj_ are indexed by J° and the nonzero 

columns of XiXq are indexed by J. Hence = XiXq^'’'^\ The submatrix p(-P) has no more than s nonzero 

rows because Xi has s nonzero rows. 

We prove Ai G [Ag]^ by contradiction. Suppose that Ai ^ [Ag]^- By (131 and (141, the entrywise ratio 
7 = Ag./Ai ^ r(P), which means that -^F*^, the first column of the circulant matrix P (as in ([T^), has 
more than one nonzero entry. Denote the indices of the first two nonzero entries of pl'd) by ki and ^ 2 - By the 
structure of circulant matrices, the rows of p('P) indexed by the following two sets (interpreted modulo n) are 
nonzero: 


Ki — {ki -I- ji — l,ki + j2 — 1, - ■ • ,ki+ jg — 1}, 

K2 = {k2 + ji - 1,^2 -I-J2 - I,” • ,^2 + js - !}• 


Note that \Ki\ = \K 2 \ = s. Recall that pl'P) has no more than s nonzero rows, hence Ki = K 2 . It follows that 


set Ki is periodic with period ^= \k 2 — ki\. By the property in Remark 4.2 the set J is also periodic with the 
same period, and we reach a contradiction. Therefore, the assumption that Ai ^ [Ag]^ is false, and Condition 1 


22 






of Corollary |2.9| is satisfied - the vector Aq is identifiable up to the transformation group. 


□ 


Corollary 4.4. If N > s, then the conclusion of Theorem \4-3\ holds for almost all Aq S C", and almost all 
Xq G that has s nonzero rows and non-periodic joint support. 


Proof. Almost all Aq S C” are non-vanishing. If > s, then almost all Xq G with s nonzero rows have 

rank s. In addition, the joint support of Xq is not periodic. Therefore, the conditions in Theorem |4.3| are met, 
and (Ao, Aq) is identifiable up to the transformation group .tF defined in (141. □ 


Corollary |4.4| shows that, in the BGPC problem with DFT matrix and a joint sparsity constraint, given that 
N > s, the identifiability of generic signals (Aq, Ag) hinges on the joint support of Xq. If the joint support is 
non-periodic, (Aq, Alo) is almost always identifiable. Other priors may imply non-periodicity. For example, if the 
joint support is a contiguous block, or if n and s are coprime, the joint support has to be non-periodic. 

Corollary 4.5. If N > s, then the conclusion of Theorem \4-3\ holds for almost all Aq G C", and almost all 
Xq G that has s nonzero rows that are contiguous. 

Corollary 4.6. If N > s, and n and s are coprime, then the conclusion of Theorem \4.3\ holds for almost all 
Aq G C”, and almost all Xq G that has s nonzero rows. 


Clearly, the coprimeness condition in Corollary |4.6| is satisfied for all s < n if n is a prime number. 

The above results are under the assumption that the sparsity level s is known a priori. If s is unknown, 
instead of solving the feasibility problem (PI), one can solve the optimization problem (P2). We have the 
following corollary, whose proof is almost identical to that of Theorem |4.3| 


Corollary 4.7. In the BGPC problem with DFT matrix and unknown sparsity level, the pair (Aq, Xq) G C" xHx 


is the unique minimizer of (P2) up to the transformation group ST defined in (141, if the following conditions 
are met: 


1. Vector Aq is non-vanishing. 

2. Matrix Xq has rank equal to the number of nonzero rows. 

3. The joint support of the columns of Xq is not periodic. 

We can derive row sparsity minimization analogs of Corollaries |4.4[ |4.5| and |4.6| in a similar fashion. These 
results are omitted for the sake of brevity. 


4.2.2 Necessary Condition 

Given that Aq is non-vanishing, “the joint support of the columns of Xq is not periodic” is necessary. We prove this 
by contraposition. We assume that the joint support of the columns of Xq is periodic with period i, and next show 


that (Ao, Aq) is not identifiable up to the transformation group in (14). Let P be a circulant matrix whose first 
column has two nonzero entries pC4) = p(^-i-i.i) _ 2 . Thus, the DFT 7 = ^/nFP^'I'> of the first column 


of P is non-vanishing. Let Ai = A 0./7 and Xi = PXq. Then P satisfies (15), and diag(Ai)FAi = diag(Ao)PA'o. 
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Since P is not a generalized permutation matrix, Xi is not a scaled and circularly shifted version of Xq. Hence 


(AojXq) is not identifiable up to the transformation group in (14l. 


The above necessary condition does not address the sample complexity. Like Proposition |3.5| we have the 
following necessary condition for the sample complexity. 


Proposition 4.8. In the BGPC problem with DPT matrix and a joint sparsity eonstraint, if (Ao,Xo) 
non-vanishing, Xq has at most s nonzero rows) is identifiable up to the transformation group in ( |14[ ), then 
N > 

— n—s 


Proof. The matrix Xq has at least n — s zero rows. If we know the locations of n — s zero rows, the problem 
becomes a BGPC problem with a subspace constraint. The columns of AXq reside in an s-dimensional subspace. 

N < ^^5^, the pair (Ao,Xo) is not identifiable up to scaling and circular shift. The proof is almost identical to 
that of Proposition |3.5[ 

The pair (Ao,>^o) cannot be identified even if we know the locations of n — s zero rows. Hence it is not 
identifiable without knowing the locations of zero rows. □ 


The above necessary condition gives a tight lower bound on sample complexity. Morrison et al. showed 
the same necessary condition for SAR autofocus (in the case of known row support of Aq). The two sample 


complexities, A > s, as is required by Theorem 4.3 implicitly and Corollary 4.4 explicitly, and N > coincide 


when s = lors = n—1. The gap between the sufficient condition and the necessary condition is analyzed next. 


4.2.3 Gap Between the Sufficient and the Necessary Conditions 

The sample complexity N > s in the sufficient condition and the sample complexity N > ^(5^ ia the necessary 
condition can be represented by the regions to the right of the line segment and the hyperbola (cf. Figure]^. 



Figure 2: The sample complexities for BGPC with DFT matrix and a joint sparsity constraint, and 
the ratio of identifiable pairs generated randomly. 


Algorithm 2 can be used to check the identifiability of BGPC with DFT matrix and a joint-sparsity constraint. 
Given Y = diag(Ao)F'Ao that has no zero rows and joint support of Xq that has cardinality s. Algorithm 2 returns 


a boolean value indicating whether or not (Aq, Ag) is identifiable up to the transformation group in (14l. The 
procedure enumerates all joint supports of cardinality s. 


24 









Algorithm 2 Identifiability of the BGPC problem with DFT matrix and a joint sparsity constraint 
input: V, the joint support J output: identifiability of (Aq, Aq) 

for all support J' such that | J'| = s do 


[diag(y(-i))]*F(--^'') [diag(y(-2))]*^{:,J'-) ... [diag(y(-^))]*F(-'^'") 

if rank(Gj') < n — 2 then 
return False 
end if 

if rank(Gj') = n — 1 and J' is not a shifted version of J then 
return False 
end if 
end for 
return True 


Proposition 4.9. Given Y = diag(Ao)FAio that has no zero rows and the joint support of Xq that has cardinality 
s, the pair (Aq, Aq) is identifiable (up to the transformation group in if Algorithm 2 returns True, and not 

identifiable otherwise. 


Proof. The matrix Gji G satisfies that Gj/x = vec{F^'A bdiag(a;)y), where F^'-A )* is an annihi- 

lator of the column space of F^' A ). Given Y that has no zero rows, any solution to the BGPC problem (A, A) 
satisfies that A is non-vanishing, and that the entrywise inverse of A is a null vector of Gji, where J’ is the joint 
support of X. On the other hand, any null vector of Gj/ produces a solution (A, A), where A is supported on 
J'. 

If Algorithm 2 returns False, then at least one of the following two cases happens: 


1. rank(Gj') < n — 2 for some \ J'\ = s. By the proof of Proposition 3.5 the solution is not identifiable even 
if the support J' is known. 


2. rank(Gj') = n — 1 for some J' that is not a shifted version of J. There exists a solution (A, A), for which 
A i [Ao]«. Therefore (Aq, Aq) is not identifiable. 


In either case, (Aq, Ag) is not identifiable up to the transformation group in (14l. 

If Algorithm 2 returns True, then rank(Gj') > n — 1 for all J' of cardinality s, and rank(Gj') = n—1 only if 
J' is a shifted version of J. Hence any solution (A, A) must satisfy that the joint support J' is a shifted version 
of J. Now, given any shifted joint support J', there exists a solution (Aj',Aj/) G [(Ao,Ao)] 5 ^. Therefore Gji 
has at least one null vector xji, which is the entrywise inverse of Aj'. Hence rank(Gj') = n — 1, and the null 
vectors of Gjt reside in the one-dimensional subspace spanned by xj'. It follows that given the joint support 
J', A in any solution must be a scaled version of Aj/. Therefore A G [Aj']^ = [Ag]^. On the other hand, given 


non-vanishing Ag, diag(Ag)F has full rank and the recovery of Ag has to be unique. Hence, by Corollary 2.9 


(Ag, Ag) is identifiable up to the transformation group in (14l. 


□ 
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The sufficient condition in Theorem |4.3| is not necessary, as shown by the following numerically constructed 
counter-examples. Let n = 10, 1 < s < 9, and 1 < < 9. The joint support J of the columns of Xq G 

is chosen uniformly at random. The entries of Aq € K" and the nonzero entries of Xq are generated as iid 
Gaussian random variables X(0,1). We use Algorithm 2 to determine whether (Aq, ATo) is identifiable up to the 
transformation group in ( |14| . For every value of s and N, and every support J of cardinality s, the numerical 
experiment is repeated independently. The ratio of identifiable pairs as a function of (s, N) is shown in Figure]^ 
When < N < s (between the line and the hyperbola), the ratio of identifiable pairs is nonzero. Therefore, 
N > s is not necessary. 

The necessary condition in Proposition |4.8| is not sufficient. This too can be demonstrated by Figure The 
ratio of identifiable pairs is less than 1 in some regions to the right of the hyperbola. Unidentifiable examples of 
(Ao, ATo) that satisfy the necessary condition can be found in Appendix [P] 

As shown by Figure when N < ^(5^ (to the left of the hyperpola), the pairs are not identifiable. When 
N > s (to the right of the line segment), the identifiability hinges on the joint support of the columns of Xq. 
Most supports are not periodic, hence most pairs are identifiable. When ((5^ < N < s (between the line and 
the hyperbola), the situation is more complicated. Besides periodic supports, other joint supports of Xq can 
also cause non-identifiability. However, given some “good” joint support of Xq that depends on both s and 
N, a randomly chosen (Aq, A^o) is identifiable almost surely. Recall that non-periodicity of the joint support is 
necessary, hence “good” supports are a subset of non-periodic supports when ((jA < N < s. For example, when 
s = 5 and N = 2, about 60% of the non-periodic supports are “good”. When s = 7 and N = 3, there is no “good” 
support. When s = 7 and N = 4, all non-periodic supports are “good”. We have the following conjecture: 

Conjecture 4.10. In the BGPC problem with DFT matrix and a joint sparsity constraint, if N > then 

for almost all Aq G C" and almost all Xq G that has s nonzero rows and some “good” joint support, the 


pair (Ao, Aq) is identifiable up to the transformation group 77 defined in (14). 


4.2.4 Extensions of the Model 


The results in Section |4.2| apply to A = F. This corresponds to MBD where the multiple channels are jointly 
sparse in the standard basis. Since the product of two circulant matrices is still a circulant matrix, we can 
easily show that the above results also apply to A = FC, where C is a known invertible circulant matrix. This 
corresponds to MBD where the multiple channels are jointly sparse in the basis formed by the columns of C. In 
fact, results such as Theorem |4.3| can also be derived for other matrices. In Section |4.3| we derive a sufficient 
condition for the identifiability of piecewise constant signals. 

Although the results in Section |4.2| deal with ID circular convolutions, extensions to higher-dimensional 
circular convolutions are straightforward. Let us consider a 2D MBD problem with a joint sparsity constraint 


as an example, and present a sufficient condition analogous to Theorem 4.3 Here A = F 0 E G C"^" is the 
2D DFT matrix, where F G is the ID DFT matrix. In the 2D problem, the row index of A can be 

represented by a pair of vertical and horizontal indices. For example, the j-th row of A corresponds to the 
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following index pair: 




where [-J denotes the floor operation. Repeating the procedure in Section 4.1 the transformation group for the 
2D problem is defined by: 


r(F (g) F) = {7 = cri/n(F (g) : ct S C is nonzero, fc G {1, 2, •••, n}}. (17) 

S' = {T : T(A, X) = (A./ 7 , (F (g F)* diag( 7 )(F (g F)X) for some 7 G r(F (g F)}. (18) 

An equivalence transformation T & S' maps X into a scaled 2D circular shift version of itself. The periodicity 
is defined as follows: 

Definition 4.11. The index set J = {(j'l , jj^), (il i J 2 )i ’ ’ ’ AJsSs)} C {1,2,-- - , ** ^^.id to be periodic 

with period £^ being integers such that 0 < £'"o-t least one of the two integers is 

nonzero), if J = {{jf + £'",ji + £^), (jl + ^”,12 + ’ ’ ’ Afs + (modulo {^/n, yjn)). 

For example, if yjn = 6 , then the index set {(1,1), (1,4)} is periodic with period (0,3). The index set 
{(1,1), (4,4)} is periodic with period (3,3). The index set {(1,1), (4,1), (1,4), (4,4)} is periodic with period 
(3,0), (0,3), or (3,3). The index set {(1,1), (5, 3), (3, 5)} is periodic with period (4,2) or (2,4). The last two 
examples are shown in Figure 



Here is the sufficient condition for the 2D problem, whose proof is almost identical to that of Theorem |4.3 


Theorem 4.12. In the BGPC problem with 2D DFT matrix F F G C" and a joint sparsity constraint at 


sparsity level s, the pair (Aq, Alg) G C" x Qx is identifiable up to the transformation group S defined in (18 1 if 
the following conditions are met: 


1. Vector Aq is non-vanishing. 

2. Matrix Xq has exactly s nonzero rows and rank s. 

3. The joint support of the columns of Xq, represented in the index pair form, is not periodic. 
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4.3 Identifiability of Piecewise Constant Signals 


Define the finite difference matrix D G and its inverse as: 


1 



1 


-1 1 


II 

7 

1 1 


1- 

-1 1 


1 1 • 

• 1 


A piecewise constant signal u can be sparsified by the finite difference operator D. Equivalently, u has the 
representation u = D~^x in which x is sparse. If C/ = D~^X in which the columns of X are jointly sparse, then 
the columns of U are piecewise constant and the discontinuities are at the same locations. 

In this section, we consider the following blind deconvolution problem. The observation model is E = 
diag(Ao)E'iD“^Ao, where the matrix Xq has at most s nonzero rows. The non-vanishing vector Aq is the DFT of 
the filter. The columns of D~^Xo are the signals, which are piecewise constant and share the same discontinuities. 
An example is deblurring of hyperspectral images. The recovery of (Aq, Xq) is the BGPC problem with A = FD~^ 
and a joint sparsity constraint. 

First, we need to figure out the ambiguity transformation group. The structured matrix P = A~^ diag( 7 )A = 
DF* di&g{-f)FD-^ = DCD-^ is 


w 

1_ 


cU) 

=2 ^ 



qA) 

0 

c(i) 


£;(") _ 

c("-l) - c(2) . 

■ c(3) - c(2) 

0 

c(2) 

_c(3) 

c(l) _ c(3) 

c(") _ c(3) 

. c(4)-c(3) 

0 

C(3) 

_c(4) 

c(A — qA) 

c(i) — 

. c(5)-c(4) 

0 

c(n-l 

- 

c("-2) _ (fn) 

c("-3) _ c(”) . 



where C = F* diag( 7 )F is a circulant matrix whose first column is 


^F*7 = c= [c(i\c(2),... 


For P to be a generalized permutation matrix, we must have = ... = = 0, and ^ 0. Hence 

7 = ^JnFc = , I]"*". The ambiguity transformation group in (121 becomes (|^. We only allow an 

unknown scaling in the recovery. 

Next we investigate identifiability up to scaling within the framework of Section and derive a sufficient 
condition. As in Theorem |4.3| one of the requirements is in terms of the joint support of the columns of Xq. We 
need the following definitions to state this sufficient condition. 


Definition 4.13. Let the index sets Ji, J 2 , ■ ■ ■ , Jt be the nodes of an undirected graph. There is an edge between 
Jt^ and .Jt2 (i < ti < t 2 < T) if Jt^ H '^*2 7 ^ The index sets Ji, J 2 , • ’ ’ ^ Jt cltc said to be connected if the 
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above graph is connected. 


Definition 4.14. The index set J = {ji,j 2 ,''' jjs} C {1,2, ••• ,n} is said to he “friendly” if for any 0 < 
ki < k 2 < ■ ■ ■ < kn-s ^ n — 1, the circularly shifted index sets Ji, J2, - ‘ ‘ ) Jn-s, defined by Jt = {ji + kt,j 2 + 
kt,‘ ‘ ‘ ijs + kt} (modulo n), satisfy that 

1- \vrajt\>n-i. 

2. Ji, ^ 2 ; • ■ ■ j Jn-s O'i'S- connected. 

We make the convention that {1, 2, • • • ,n} is friendly. 

If the index set J is friendly, and the entries indexed by its circularly shifted version Jt(l<t<n — s) 
are equivalent in some sense, then due to transitivity of the equivalence relation, and the connectivity of the 
circularly shifted index sets, at least n — 1 out of n entries are equivalent. This property is used in the proof of 
Theorem 14.201 


Remark 4.15. If the index set J is friendly, then its flipped and shifted versions are also friendly. 


We have the following propositions regarding the “friendliness” of an index set. Proposition |4.16| shows that, 
for a non-trivial problem, a friendly index set must have cardinality at least 3, which helps to avoid degeneracy 
in the proof of Theorem |4.20[ Propositions |4.17| and |4.18| give two sufficient conditions for friendliness, which 


makes the property more readily interpretable. Corollary 4.19 gives an alternative characterization of Condition 
1 in Definition |4.14[ See Appendix [E| for the proofs. 


Proposition 4.16. If n> A and the index set J is friendly, then | J| >3. 

Proposition 4.17. The index set J is friendly if \J\ >3 and J is contiguou^ 

Proposition 4.18. The index set J is friendly if | J| > ^ and J is not periodic. 

Corollary 4.19. Let | J| = s < n. Then \ — \ for all choices of n — s shifted index sets Jt if and 

only if J is not periodic. 


Here is the sufficient condition for identifiability of piecewise constant signals. 

Theorem 4.20. Consider the BGPC problem with A = FD~^ and two constraints: X is non-vanishing, and 
the columns of X are jointly s-sparse. The pair (Aq, Aq) S C” x Tlx is identifiable up to an unknown scaling, 
if the following conditions are met (assume that n > 4 and J = {ji,j 2 ,''' ,js} denotes the joint support of the 
columns of Xq): 

1. The vector Aq is non-vanishing. 

2. The matrix Xq has exactly s nonzero rows, and has rank s. 

3. \iJ. 

4- is friendly. 

®Index sets like {n, 1, 2} are considered contiguous due to the circularity. 
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Proof. First, given non-vanishing Ao and A = FZ? the matrix diag(Ao)FI? ^ has full rank. If diag(Ao)FI? 


diag(Ao)FiA“^Xi, then Xi = Xq. Hence, given Aq, the recovery of Xq is unique. By Corollary 2.9 to establish 
the result, we only need to show that Aq is identifiable up to an unknown scaling. 

Assuming that Conditions 1-4 of the theorem are satisfied, we show that Ai is a scaled version of Aq, if 
diag(Ao)FiA“^Ao = diag(Ai)FIZ“^Ai for (Ai,Xi) that satisfies the two constraints. The matrix diag{Xo)FD~^ 
has full rank, hence both Xq and Xi have rank s. Without loss of generality, we may assume that Xq and Xi 
only have s columns, which are linearly independent, by removing redundant columns at the same locations in 
both matrices. They both have full column rank s and exactly s nonzero rows. By assumption, the vectors Aq 
and Ai are non-vanishing. Write Xi in terms of Xq, Xi = PXq, where 


P = ZAF*[diag(Ai)] ^ diag(Ao)FIZ = DF* diag{^)FD 


- 1-1 


The matrix P has the structure in (191 where c = Furthermore, P satisfies (161 in 

The submatrix p(-A) = XiX^^ ''^^ has at most s nonzero rows and at least n — s 


the proof of Theorem 


4.3 


zero rows. The submatrix ^ _ x zero rows. We denote the corresponding index set by 

K = {ki,k 2 ,“ ■ , kn-s-i}. By (191, the row jg- 


p{k,J) _ g(fc+i-ii) _ g(fc)^ g(fe+i-i2) _ ^ g(fc-l-i-td _ g(fc) 


The index set Jj. = {k, k + 1 — ji,k + 1 — j 2 , - ■ ■ ,k + 1 — jg} is a flipped and shifted version of {1} IJ — 
{1) Ji) J 2 , • • • ) js}- The above row p(^’P is zero, which means all the entries of the subvector are equal. By 

the assumption that {1}1J J is friendly, the index sets Jfci, • • • , axe connected. That means all the 

entries of c indexed by equal. Besides, iur=r'T,j > n — 1. That means either all the entries 

of c are equal or there is one entry with a different value. There are three different cases: 

1. All the entries of c are equal. Then the vector Ag./Ai = ^JnFc has n — 1 zeros, which contradicts the 
assumption that Ao,Ai are non-vanishing. 

2. All but the /co-th entry of c are equal, where /cq yf 1. Then all the entries of ^i^at do not contain 

g(fco) gj-g 2 gros, and all the entries that contain are nonzeros. The rows indexed by K are zeros, hence 
they do not contain The row indexed by /cq is shown in (20), and is nonzero. The rows that contain 


any of the s entries in (21) are also nonzeros. 


g(fco-ii-l-l) _ g(fco) ffko-ji + l) _ g(feo) 


g(feo-is-l-l) _ g(fco) 


( 20 ) 


g(fco) _ g(fco-l-il-l) g(fco) _ g(feo+f2-l) 


g(feo) _ ^(ko+js-d 


( 21 ) 


Note that no two entries in (21) can belong to the same row; no entry in (21) belongs to the row in (20). If 


every entry in (21) belonged to a row in there would be s-1-1 nonzero rows in p(2:n,j) xhe number 


of nonzero rows in pl'A) jg at most s. Hence, one of the s entries in (21) is not in any row of p(2:"P), gy 
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observation, the only entry that could be missing is Assume that, without loss of generality, 

c(fco) _ (.(feo+ii-i) jg ajjy j-Q^ Qf p(2:ra.J)^ That implies ko + ji — 1 = 1 (modulo n). Hence there 

exists an entry in the first row ~ ^i =2 n > 4 and {1} IJ J is friendly, by 

I J| > 2. Hence there exists another entry in the first row Since 


Proposition 


4.16 


there are s nonzero rows in p( 2 :n,J)^ |.]^g p(iA) jn^gt be zero. Hence, 


ko n+ 2 -j 2 

g(t) _ g(i) _ 


1=2 1=2 


Recall that all the entries of c are equal except for c^^°\ It follows that = ... = = 0, 

resulting in a contradiction. 

3. All but the first entry of c are equal. Then all the entries of g(i) ^re zeros, and 

all the entries that contain are nonzeros. In particular, the entries • • • , 

in the rows indexed by tJs are nonzeros. Hence the first row must be zero. Therefore, 

£;(2) = (;(3) = . . . = (Xn) — g^ud ^ 0. 

The only case that does not cause a contradiction is the third, which leads to c = 0,0, • • • , 0]"'" and Aq./Ai = 

y/nFc = 1, • • • , 1]"'’. Therefore, Ai = ^Aq is a scaled version of Aq. □ 


A result for generic signals, analogous to Corollary |4.4| follows immediately. 

The requirement N > s, implied by Theorem |4.20[ is not necessary. We have the following necessary 


condition, which can be proved similarly to Proposition 4.8 


Proposition 4.21. In the BGPC problem with A = FD ^ and a joint sparsity constraint, if (Aq, Aq) (^Aq is 
non-vanishing, Xq has at most s nonzero rows) is identifiable up to scaling, then N > 

An analysis of the gap between the sufficient and the necessary conditions, similar to Section can be 

carried out for these results too. It is omitted for brevity. 


5 Universal Sufficient Condition for BGPC with a Sparsity Constraint 

In this section, we consider the BGPC problem with a sparsity constraint on the total number of nonzero entries 
in the matrix X, denoted by ||A||o. Consider the following problem: 

(P3) find (A,A), 

s.t. diag(A)AA = Y, 

A G C", X Gnx = {X G : ||A||o < s}. 

The measurement is A = diag(Ao)AAo. We only consider the case where A G C"^" is an invertible square 
matrix. The vector Aq G C" is non-vanishing. The matrix Xq G has at most s nonzero entries. 
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The ambiguity transformation group associated with the matrix A is the same as in Section 4.1 In 
Theorem |5.1| we show that Xq is identifiable up to a generalized permutation in the ambiguity transformation 
group associated with A if the rows of Xq form the most sparse basis of its row space. This is a universal 
sufficient condition for BGPC with a sparsity constraint, which applies to every invertible square matrix A. This 
universal result is derived using the general framework in Section 


Theorem 5.1. In the BGPC problem with a sparsity constraint at sparsity level s, the pair (Aq, Xq) is identifiable 
up to the ambiguity transformation group IX associated with A, if the following conditions are met: 


1. Vector Ao is non-vanishing. 

2. If an invertible matrix P S satisfies that ||PXo||o < ll-^o||o; then P is a generalized permutation 

matrix. 

3. ||Xo||o = s. 


Proof. Given non-vanishing Aq and invertible A, the matrix diag(Ao)^ is invertible. Hence given Aq, the matrix 


Xq is identifiable. By Gorollary 2.9 we only need to show that Aq is identifiable. Suppose that diag(Ao)^Afo = 
diag(Ai)HXi and ||Xi||o < s = ||Xo|jo- By the above Gondition 2, Xq has full row rank n. Otherwise, there exists 
an invertible matrix P that is not a permutation matrix and satisfies PXq = Xq, which clearly violates Gondition 
2. The matrix diag(Ao)^ is invertible, hence rank(Xi) = rank(Xo) = n. There are no zero rows in AXq or AXi. 
Hence Ai is also non-vanishing. Write Xi in terms of Xq, Xi = PXq, where P = H“^[diag(Ai)]“^ diag(Ao)T. 


By the above Gondition 2, P has to be a generalized permutation matrix. By (111 and (12|, 7 = Aq./Ai G r(H) 
and Ai G [Ao]^- Therefore, Aq is identifiable. □ 


If the sparsity level is not known a priori, we can solve the following optimization problem (P4). Under the 
above Gonditions 1 and 2, the minimizer in (P4) is unique up to the same transformation group. If the minimizer 
to (P4) has sparsity s, then it is the solution to (P3) as well. 


(P4) min Iloilo, 

(A,A) 

s.t. diag(A)HAf = Y, 

A G c”, a: G 

The following universal sufficient condition follows by combining Theorem |5. 1 1 with results about the distribution 
of non-zero elements in random matrices and in the products of such matrices with vectors |16| . 

Theorem 5.2. Suppose that the vector Aq is non-vanishing, the matrix Xq G is Bernoulli-Gaussian 

random matrix, where Xq = BqG, the entries of B are iid Bernoulli random variables B(l,9), and the entries 
of G are iid Gaussian random variables N{0, 1). If ^ < 9 < ^ and N > Cnlogn for a sufficiently large absolute 
constant C, then the pair (Aq, AIq) is identifiable in (U4^, up to the ambiguity transformation group IX associated 
with A, with probability at least 1 — exp(—c0iV) for some absolute constant c. 
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Proof. We prove the identifiability by showing that Condition 2 in Theorem |5.1| is satisfied with probability at 
least 1 — exp(—c0iV) given the above Bernoulli-Gaussian model. Assume that P G is an invertible matrix 

but not a generalized permutation matrix. Since P is invertible, there exists a permutation of 1, 2, • • • , n, denoted 
by ji) j 2 ) • • • ,jn, such that the support of the fcth row contain the index jk, i.e., ^ 0 , for 1 < fc < n. 

Since P is not a generalized permutation matrix, there exists at least one row with more than one nonzero entries. 
If the row p(^'-) has only one nonzero entry p('^dfc)^ then ||(PAq)^^’’^|| o = ||^’^^’■^Ao||o = II||o. Next, we 
show that if P^^’-) has more than one nonzero entries, then ||(PAo)^^’')||o > ||Ag'^'”’'^||o with high probability. 


By Lemma 17 in 16 , if the Bernoulli-Gaussian matrix Ag satisfies that ^ < (^ < \ and N > Cnlogn for a 
sufficiently large constant C, then the probability that there exists a vector v G C" with more than one nonzero 
entries such that ||u*Ag||g < is at most exp(—ci 6 >A), for some absolute constant ci. Therefore, with 

probability at least 1 — exp(—ci 0 A), 


K^^o) 


(k/.: 


lo > jCN 


( 22 ) 


for every index k such that p(*^'-) has more than one nonzero entries. 

By Lemma 18 in |16| , the probability that any row of the Bernoulli-Gaussian matrix Ag has more than 
^6N nonzero entries is at most nexp(—0A/243). Since N > Cnlogn for a sufficiently large constant C, the 
probability n exp(—0A/243) < exp(—C 20 A) for some absolute constant C 2 . Therefore, with probability at least 
1 - exp(-C 26 'A), 

2 ■ ^ in 

(23) 




for every k. 

Combining (22) and ( [2^ , || (PAg)^^’-^||g > ||Ag'^'"’'^||g for every index k such that has more than one 

nonzero entries, with probability at least 1 — exp(—c0A) for some absolute constant c. Therefore, with the same 
probability, 

n n 

llPAollg = ^ ||(PAg)('=-^)||g > ^ HA^^-^^Hg = ||Ao||g. 


/c=l 


/c=l 


Equivalently, Condition 2 in Theorem 5.1 is satisfied with probability at least 1 — exp(—c^A^). 


□ 


6 Discussion 

We defined identifiability of a bilinear inverse problem up to transformation groups. A general framework 
for proving identifiability was proposed. The framework was applied to the problem of BGPC. We showed 
sufficient conditions for unique recovery up to a transformation group under three scenarios, with a subspace 
constraint, with a joint sparsity constraint, and with a sparsity constraint, respectively. We also provided 
necessary conditions for the scenarios with a subspace constraint or a joint sparsity constraint. We developed 
a procedure to determine the ambiguity transformation groups for BGPC with joint sparsity or with sparsity 
constraints. We also designed algorithms that can check the identifiability for BGPC with subspace or with joint 
sparsity constraints, and demonstrated the tightness of our sample complexity bounds by numerical experiments. 

The analysis in this paper is not always optimal. In certain cases, there exist gaps between the sufficient 
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conditions and the necessary conditions. For example, in the scenario with DFT matrix and a joint sparsity 
constraint, the gap between the sample complexities in the sufficient and the necessary conditions is TV > s versus 
TV > However, we believe that it would be possible to bridge these gaps by introducing more stringent 

assumptions (e.g., generic vectors and matrices). 

One goal of this paper is to motivate more research into the identifiability of bilinear inverse problems. 
For BGPC, additional identifiability results can be obtained for different bases A and different constraint sets 
Oa, For example, exploiting the extra information regarding A (positivity in inverse rendering, unit-modulus 
entries in SAR autofocus), is expected to provide less demanding conditions for identifiability. The merit of the 
framework in this paper for identifiability in bilinear inverse problems is not restricted to the demonstrated 
exemplary applications. It will be useful for analyzing a wider class of practical applications, including blind 
deconvolution (with a single channel and/or the linear convolution model), phase retrieval, dictionary learning, 
etc. 


A Example of a Non-trivial Annihilator 

Most bilinear mappings that arise in applications do not have non-trivial left or right annihilators, however this is 
not universally true. Here is an example in which the bilinear mapping does have a non-trivial right annihilator. 
Assume that z = in the following BIP: 

find (x,p), 
s.t. xp^^^ = z, 

X €C^, y G C^. 


Then (xo,yo) is identifiable up to the following transformation group: 


^ = I^T : T{x,y) = x, [ay^^\y^'^'^ + some nonzero tr S C and some r e c| . 

Let T = {Tx,Ty), where Tx{x) = ^x, Tyijj) = -I- r]"'". Note that Ty is not a linear transformation if 

r 7 ^ 0. In addition. Condition 2 in Corollary 2.9 is not necessary. Civen J-{xo, yo) = J-{xo, y), i.e., Xoyg^^ = xoy^^\ 
it is not necessary that y = yo- The reason is that the bilinear mapping J- has a non-trivial right annihilator 

?/ = [o,iF. 


B Proof of Lemma 3.2 


1. If A S has full row rank, then the rows of A form a basis for TZ{A) whose dimension is n. For every 

non-empty proper subset J and its complement J°, TZ{A^'^'-'>) and TZ{A^'' ’■)) are two subspaces whose 
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dimensions are | J| and \ J^\ respectively. Therefore, 

n[A) = 

dim(7^(^)) = n = I J| + | J"| = dim(7^(A(■^-))) + dim(7^(y4('^“-))). 

Therefore, the sum of two subspaces is a direct sum, and the row space of A is decomposable. 

2. If the row space of A is not decomposable, then A does not have full row rank. If the matrix A has full 
column rank, then n > m. 

Next, we prove n > m by contradiction. Suppose that n = m. Since square matrix A has full column rank, 
it must have full row rank, which causes a contradiction. Therefore, the assumption is false, and n has to 
be greater than m. 

3. The row space of A is not decomposable, if and only if the sum TZ{A) = TZ{A^'^''-'>) + 71{A^'^ ’■)) is not a 
direct sum for any non-empty proper subset J C {1, 2, • • • , n}, or equivalently, 

dim(7^(A)) < dim(7^(yI(“'-))) -kdim(7^(A(“'"-))), 

for all non-empty proper subsets J C {1, 2, • • • , n}. 


C Examples of Ambiguity Transformation Groups 


In the BGPC problem with a joint sparsity constraint, the ambiguity transformation groups for A can be figured 
out with the method in Section 4.1 The ambiguity transformation groups associated with A = F and A = FD~^ 


are shown in Section [4~T] and Section [41^ respectively. We give more examples here. 

The matrix A introduces some “mixing” to the rows of X. li A = I, there is no mixing. The structured 
matrix I~^ diag( 7 )/ = diag( 7 ) is a diagonal matrix. It is a generalized permutation matrix provided that 7 is non¬ 
vanishing. The set of 7 which produces a generalized permutation matrix is r(/) = {76 C" : 7 is non-vanishing}. 
The ambiguity transformation group is 


^ = {T : T{X,X) = (A./ 7 , diag( 7 )X) for some non-vanishing 7 }. 


In this case, any non-vanishing A is considered equivalent to Aq. The identifiability of (Ao,Xo) with this trans¬ 
formation group is not an interesting problem. 

For some A, the structured matrix A~^ diag( 7 )A is already studied in the literature. For example, if A 
is a DFT matrix, A~^ diag( 7 )A is a circulant matrix. If A is the discrete cosine transform (DCT) matrix. 


A ^ diag( 7 )A is the sum of a symmetric Toeplitz matrix and a Hankel matrix 23 . For other matrices, the 


structure oi A ^ diag( 7 )j 4 can be figured out by symbolic computation. The matrix A = FD ^ in Section 4.3 


is an example. Another example is the Haar matrix i/„, corresponding to a wavelet transform. The matrix Fl^ 
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and the structured matrix ^ diag( 7 )i ?4 are 


Hi = 


1111 
11 -1-1 
1-10 0 
0 0 1-1 


Hi diag(7)i/4 = ^ 


^( 1 ) _^( 2 ) 


yi) _^( 2 ) 


^( 1 ) _ ^( 2 ) 


^( 1 ) _ ^( 2 ) 


^(1) + ^(2) + 27(3) ^(1) +^(2) _ 27(3) 

^(1) + ^(2) _ 27(3) 7(1) +7(2) +27(3) 

7(1) -7(2) 7(1) + 7(2) 27(4) 7(1) + 7(2) _ 27(4) 

7(1) -7(2) 7(1) + 7(2) _ 27(4) 7(1) + 7(2) 27W 


7(1) _ 7(2) 


7(1) _ 7(2) 


The structured matrix ^ dia,g{'j)Hi is a generalized permutation matrix if and only if 7 ^^^ = 7 ^^\ 7 *-^^ = ± 7 *' 4 ) 
and 7 *^ 4 ) = ± 7 ( 1 ). The set r(_ff 4 ) and the ambiguity transformation group ^ are 

r( 7 J 4 ) = {7 : 7 *'^^ = 7 ^^^ = ( 7 , 7 ^^^ = ±( 7 , 7 ^"*^^ = ±( 7 , for some nonzero a S C}, 


= {T T{XjX) = (A./ 7,-^4 ^ diag( 7 ) 7 J 4 X) for some 7 S r(i/ 4 )}. 


D Insufficiency of the Condition in Proposition |4.8| 

The necessary condition in Proposition |4.8| is not sufficient, even when the locations of the zero rows are known 
a priori. For example, when n = 7, s = 4, Aq G is non-vanishing and 


1 


^0 = 


0 

0 

0 

0 

0 

0 


0 0 

1 0 

0 1 

0 0 , 

0 0 

0 0 

0 0 
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the pair (Ao,Xo) is not identifiable, even if we know that the last three rows of Xq are zeros. There exists a 
cirulant matrix P whose first column is [1, 2, 0,0,0,0, 0]"’’ such that 


1 0 0 
2 1 0 


0 2 1 


= PXo = 


0 0 2 , 


0 0 0 


0 0 0 


0 0 0 


aud Ai = A 0 ./ 7 , where 7 = y^F[l, 2 , 0 , 0 , 0 , 0 , 0 ]"^ is uou-vanishiug. 

The above example is a degenerate case where the actual joint sparsity of Xq is less thau s = 4. A non- 
degeuerate Xq may also not be identifiable, if there is no extra knowledge of the locations of the zero rows. For 
example, 

1 3 2 

2 1 3 

3 2 1 


Ao = 


-29 -28.5 -17.5 


0 0 0 
0 0 0 
0 0 0 


There exists a circulant matrix P whose first column is [2,16,1,8, 0.5,4, 32]"^, such that 


63.5 

0 


-889 


Ai = PXo = 


0 


-444.5 


0 

-190.5 


31.75 

0 

-889 

0 

-444.5 

0 

-127 


95.25 

0 

-508 

0 

-254 

0 

-63.5 


aud Ai = A 0 ./ 7 , where 7 = y^F[2, 16,1,8, 0.5,4,32]"^ is uon-vanishing. 

The above pathological examples reside in a set of measure zero. Next, we show that wheu rank(Ao) = s but 
the joint support of the columns of Xq is periodic, the pair (Aq, Xq) is not identifiable. This set of unidentifiable 
Xq has nonzero measure. Recall the proof of Theorem |4.3[ Assume that the joint support of the columns of Aq 
is periodic with period £. There exists a circulant matrix P with two nonzero entries in the first column, indexed 
by ki and k 2 , such that k 2 — ki = i and 7 = ^/nFP^'’^'> is non-vanishing. Hence there exists Xi = PXq and 


37 








Ai = Aq./t such that diag(Ao)FXo = diag(Ai)FXi and Ai ^ [Aq]^- Therefore, (Ao,Xo) is not identifiable. 


E Proof of the Propositions Regarding “Friendliness” 

Proof of Proposition [7. f ^ We prove by contraposition, i.e., if n > 4 and | J| < 2, then J is not friendly. First, 
if J = 0 or I J| = 1, then the circularly shifted index sets Ji, J 2 , • • • , Jn-s are not connected. 

Next, we show that if n > 4 and | J| =2, then Ji, J 2 , • • • , Jn-s are not connected. Since all the circularly 
shifted index sets are equivalent, without loss of generality, we may assume that J = {1, r}, where 2 < r < f + 1. 
Then all the sets {ri,r 2 } such that ri — r 2 = r — 1 (modulo n) or r 2 — ri = r — 1 (modulo n) are circularly 
shifted versions of J. There are a total of n circularly shifted index sets. 

If r = ^ + 1, then J is periodic. The sets like {'ri,ri + f} (1 < ri < are counted twice because ^ ^ 

(modulo n). And these index sets are not connected. 

If n > 4 and r < f + 1, the n index sets are {1, r}, {2, r + 1}, • • • , {n — r + 1, n}, {n — r + 2,1}, • • • , {n, r — 1}. 
By removing {r, 2r — 1} and {n — r + 2,1}, there are n — 2 > 2 index sets left. These circularly shifted versions 
of J are not connected because J = {1, r} is not connected to the rest. □ 

Proof of Proposition EH First, if J is contiguous and | J| = s, then n — s shifted contiguous index sets cover 
at least s + (n — s — l)=n—1 indices. Therefore, | UEi* '^*1 > ^ ~ 1- 

Next, we prove that the shifted index sets Ji, J 2 , -'' > Jn-s are connected by showing that they form a cycle 
or a path in the graph. To this end, we show that between the n — s pairs (Ji, J 2 ), (J 2 , J 3 ), • • • , {Jn-s, Ji), there 
are at least n — s — 1 edges. Suppose the opposite, that there are fewer edges, for example two edges are missing 
in the above cyle. Then n — s shifted contiguous index sets cover at least s + s + (n — s — 2) = n + s — 2>n+l 
indices, a contradiction. □ 


Proof of Proposition 4 ..18 We first show that if J is not periodic, then | Jt\ ^ n — 1, or equivalently 
I nrJi* \ — 1 - We prove the contrapositive, if there are two distinct indices k', k" G nr=i* then J is periodic. 
Note that Jf, J 2 , • • • , Jf_s are all circularly shifted versions of the same index set = {jf, ’ ’ ’ ) Jn-s}- 


= {ji) J 2 > • • • ) Jn-si = W - ki,k' -k2,--- ,k' - kn-s} = {k” - ki,k" - ^2, • • • , fc" - kn-s} (modulo n). 


Hence J'^ is periodic with period £ = |fc" — fc'|, so is J. 

Next we show that the shifted index sets are connected. If | J| > then Jt^ H --^*2 h)r any ti,t 2 - There 
is an edge between every pair of nodes, hence the graph is a complete graph, which is connected. □ 

Proof of Corollary \4-lS\ The sufficiency is shown in the proof of Proposition 14.18] 

Next we prove necessity. If J is periodic with period ^ and |J| = s < n, then for any k',k” such that 
k” — k' = £, we can always apply the proper shifts ki = k' — ji, k 2 = k' — • • • , kn-s = k' — jf-s such that 

k', k” £ n"=i* Ji- Hence we can pick n — s shifted index sets such that | lj"=i* Jt\ ^ n — 2. □ 
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